Image pickup apparatus and image pickup apparatus control method, and program

ABSTRACT

To provide an apparatus and method for realizing a processing with which a pixel gravity center control can be conducted when a pixel number reduction image is generated in an image pickup apparatus. A pixel unit configured to output a pixel addition signal obtained by adding output pixel signals from pixels having different sensitivities to each other and a pixel information synthesis unit configured to execute a weighted addition processing of multiplying plural pixel addition signals output from the pixel unit by a previously set gain and calculate a configuration pixel value of a pixel number reduction image where a total number of pixels is lower than a number of pixels of the pixel unit are provided. A control on gravity center positions for respective pixels constituting the pixel number reduction image is executed through the pixel addition signal generation processing by the pixel unit and the weighted addition processing by the pixel information synthesis unit. For example, a gravity center position control to evenly arrange the gravity center positions for the respective pixels constituting the pixel number reduction image in the pixel unit is executed. Through this processing, it is possible to generate a high quality image in which a generation of jaggies or the like is suppressed.

TECHNICAL FIELD

The present invention relates to an image pickup apparatus and an imagepickup apparatus control method, and a program. In further detail, theinvention relates to an image pickup apparatus that generates an imagesignal in which the number of pixels is transformed through a processingon an output signal of an image pickup element and an image pickupapparatus control method, and a program.

BACKGROUND ART

Many of image pickup apparatuses in these days are being provided withan image pickup element having an extremely large number of pixels fromseveral million to several tens of million pixels for picking up a highresolution image.

However, it is uncommon to utilize a display apparatus that can output ahigh resolution image corresponding to the number of pixels provided tothe image pickup element, and a situation also occurs in which if thehigh resolution image output from the image pickup element is recordedin a memory as it is, a memory capacity necessary for the recording isincreased, and the number of images that can be recorded is decreased.

By taking the above-mentioned circumstances into account, in the imagepickup apparatus provided with the image pickup element having the largenumber of pixels, an output pixel signal from the image pickup elementis not recorded in the memory as it is, and a processing of reducing thetotal number of pixels to be recorded in the memory is often carried outthrough a processing of thinning out the number of output pixels or asynthesis processing including an addition computation of plural pixelsand the like.

A related art technology disclosing the above-mentioned pixel numberreduction processing includes, for example, PTL 1 (Japanese UnexaminedPatent Application Publication No. 2008-278453).

With reference to FIG. 1, an outline of the processing described in PTL1 will be described.

FIG. 1 illustrates

(a) a pixel arrangement of an image pickup element, and

(b) a circuit configuration of the image pickup element

which are described in the above-mentioned PTL 1.

It should be noted that the circuit illustrated in FIG. 1( b) is acircuit corresponding to a pixel region 11 for a part of four pixelsincluding Gb1, R1, Gb2, and R2 illustrated in FIG. 1( a).

According to one of embodiments described in PTL 1, in the image pickupelement configuration illustrated in FIG. 1( a), the pixel region 11 isset as a common pixel region, that is, a unit region including pluralpixels for deciding a pixel value of one pixel constituting an outputimage as the pixel number reduction image. The pixels of Gb and R areconnected via transfer transistors (T1 to T4) to a gate part of atransistor for amplification (AMP) as illustrated in FIG. 1( b).

To obtain the output image where the number of pixels is reduced, byusing the circuit configuration illustrated in FIG. 1( b), an additioncomputation of pixels having a same color included in the region 11(according to the present example, Gb1 and Gb2) is carried out.

FIG. 2 illustrates a timing chart of a pixel value addition processingexecuted by using the circuit configuration illustrated in FIG. 1( b).

FIG. 2 illustrates these signal patterns of

transfer transistor signals: T1 to T4,

a reset signal: RST (reset of floating diffusion (FD)), and

a selection signal: SEL, in

(1) a shutter operation for regulating an exposure start, and

(2) a read out operation for regulating an exposure end.

FIG. 2 illustrates a signal pattern in a case where an additionprocessing for the pixel Gb1 and the pixel Gb2 illustrated in FIG. 1( a)is carried out.

A period between times ts and te illustrated in FIG. 2 is equivalent toan exposure period.

In response to ON of the transistors T1 and T3 in the shutter operationof (1) in FIG. 2, an exposure for the pixel Gb1 and the pixel Gb2illustrated in FIG. 1( a) is started. After that, in response to ON ofthe transistors T1 and T3 in the read out operation of (2) in FIG. 2,the exposure for the pixel Gb1 and the pixel Gb2 illustrated in FIG. 1(a) is ended, and the read out processing is executed.

In an initial stage (T=t1) of the read out processing, the selection(SEL) of the common pixel and the reset (RST) of the floating diffusion(FD) are carried out, and subsequently, at a time (T=t2), thetransistors T1 and T3 of the pixel Gb1 and the pixel Gb2 illustrated inFIG. 1( a) are read out at the same time, and electrons generated in Gb1and Gb2 are accumulated in the floating diffusion (FD) for the addition.Thus, an addition signal based on pixel values of the two pixels isobtained. An average signal of the plural pixels is calculated on thebasis of these addition signals, for example, and an output pixel valueis calculated.

According to PTL 1, while the above-mentioned addition processing forthe plural pixel is used as a basic configuration, for example, a pixelnumber reduction configuration through a pixel value addition while fourpixels are set as one pixel as illustrated in FIG. 3 is disclosed. FIG.3 illustrates these drawings of

(a) a pixel array of the image pickup element, and

(b) a pixel gravity center of the output pixel.

The pixel array illustrated in FIG. 3( a) is a pixel array similar tothe pixel array illustrated in FIG. 1( a). This pixel array isequivalent to a pixel array of the image pickup element.

On the basis of four pixels corresponding to pixels having the samecolor in the image pickup element of FIG. 3( a), a pixel value of onepixel of the output image is set and output.

That is, the total number of pixels is reduced by consolidating the fourpixels into one pixel for the output.

The pixel gravity center of output pixel of FIG. 3( b) illustrates apixel gravity center in the original image pickup element with regard toeach of the output pixels after the reduction in the number of pixels.

For example, a Gb pixel 31 of FIG. 3( b) corresponds to a pixel valuedecided by evenly using pixel values of Gb pixels on four corners in a3×3 pixel block 21 of FIG. 3( a), and a pixel gravity center is set at acenter position of the 3×3 pixel block 21. This gravity center positionis represented by the Gb pixel 31 of FIG. 3( b)

An example illustrated in FIG. 3 is an example in which a processing ofreducing the number of pixels of the output image into ¼ of the numberof pixels of the image pickup element is carried out while 64 pixels of8×8 pixels illustrated in (a) is set as 4×4=16 pixels illustrated in(b).

For this processing, for example, the addition processing on the (Gb)pixels on the four corners of the 3×3 pixel block 21 illustrated in FIG.3( a) is executed, and the pixel value of one Gb pixel in the outputimage is calculated.

The pixel values of the two Gb in a vertical direction are added whilefollowing the circuit described above with reference to FIG. 1( b).After that, two addition values output from the image pickup element inevery other column are further added to calculate an addition value ofthe four Gb pixel values, and thereafter, an average value or the likebased on these addition values is calculated to decide the pixel valueof one pixel of the output image based on the four pixel.

That is, on the basis of the four Gb pixels included in the 3×3 pixelblock 21 illustrated in FIG. 3( a), the pixel value of the Gb pixel 31illustrated in FIG. 3( b) is calculated.

In this case, the gravity center of the Gb pixel 31 in the output imageis at a position of (x, y)=(2, 2) in coordinate axes where a horizontalright direction is set as x and a vertical downward direction is set asy, that is, a position of the Gb pixel 31 illustrated in FIG. 3( b).

Also, on the basis of the four B pixels included in a 3×3 pixel block 22illustrated in FIG. 3( a), a pixel value of a B pixel 32 illustrated inFIG. 3( b) is calculated.

In this case, a gravity center of the B pixel 32 in the output image isat a position of (x, y)=(3, 2), that is, a position of the B pixel 32illustrated in FIG. 3( b).

Similarly, on the basis of the four Gb pixels included in a 3×3 pixelblock 23 illustrated in FIG. 3( a), a pixel value of a GB pixel 33illustrated in FIG. 3( b) is calculated.

In this case, a gravity center of the Gb pixel 33 in the output image isat a position of (x, y)=(6, 2), that is, a position of the B pixel 33illustrated in FIG. 3( b).

Also, on the basis of the four B pixels included in a 3×3 pixel block 24illustrated in FIG. 3( a), a pixel value of a B pixel 34 illustrated inFIG. 3( b) is calculated.

In this case, a gravity center of the B pixel 34 in the output image isat a position of (x, y)=(7, 2), that is, a position of the B pixel 34illustrated in FIG. 3( b).

The total 16 pixels illustrated in FIG. 3( b) are output as an image of4×4 pixels in a case where the pixels are set as an output image.

That is, the pixels are output as an image 70 of 4×4 pixels asillustrated in FIG. 4( c).

FIG. 4 illustrates these

(b) a pixel gravity center of the output pixel (same as FIG. 3( b)), and

(c) a pixel position of the output image.

In FIG. 4( c), a consideration is given while 2×2 pixels on the upperleft, that is, a block of 2×2 pixels including the Gb pixel 31 and the Bpixel 32 is fixed. In the case of this setting, the other three 2×2pixel blocks are all moved in accordance with arrows (α), (β), and (γ)illustrated in FIG. 4( c) and output as constitutional pixels of theimage 70 of 4×4 pixel illustrated in FIG. 4( c).

Through this shift processing, the following problem occurs.

For example,

the Gb pixel 33 where the position of the pixel gravity center is at (x,y)=(6, 2) is set as a Gb pixel 53 where the pixel position is at (x,y)=(3, 2) in the output image.

Also,

the B pixel 34 where the position of the pixel gravity center is at (x,y)=(7, 2) is set as a B pixel 54 where the pixel position is at (x,y)=(3, 3) in the output image.

Here, a reduction scale rate is calculated.

A consideration is given while it is assumed that the GB pixel 31 at thepixel position (x, y)=(2, 2) is set as a reference pixel at a fixedposition.

The Gb pixel 33 at the pixel position (x, y)=(6, 2) illustrated in FIG.4( b) is away by 4 pixels from the Gb pixel 31 corresponding to thereference pixel.

In the output image, since this is set as the Gb pixel 53 where thepixel position is (x, y)=(3, 2), a distance from the reference pixel:the Gb pixel 31 is 2 pixels.

That is, the reduction scale rate is

2 pixels/4 pixels=½.

On the other hand, the B pixel 34 at the pixel position (x, y)=(7, 2)illustrated in FIG. 4( b) is away by 5 pixels from the Gb pixel 31corresponding to the reference pixel.

In the output image, since this is set as the B pixel 54 where the pixelposition is (x, y)=(4, 2), a distance from the reference pixel: the Gbpixel 31 is 3 pixels.

That is, the reduction scale rate is

3 pixels/5 pixels=⅗.

In this manner, the reduction scale rates between the pixels fluctuate,and an output image having a relative position different from relativepositions of the respective pixels of the picked-up image correspondingto the pixel array of the image pickup element is generated.

That is, the output image is generated while spacings between therespective pixels of an original image picked up in the image pickupelement are unevenly reduced.

The above-mentioned unevenness in the pixel spacings causes an imagequality degradation.

To be specific, for example, a degradation such as an expansion ofjaggies illustrated in FIG. 5 occurs.

The original image of FIG. 5(A) is a high resolution image with a largenumber of pixels equivalent to the picked-up image of the image pickupelement, and this image has small jaggies.

If the pixel number reduction processing with which the relative pixelpositions described with reference to FIG. 1 to FIG. 4 are set to bevaried is carried out on the basis of this original image of FIG. 5(A),an image in which the jaggies are expanded as illustrated in FIG. 5 (B)is generated.

It should be noted that the jaggies are a type of folding noise. Becauseof the unevenness in the spacings between the pixel gravity centersafter the addition, the jaggy degradation is increased.

It should be noted that in addition to this, a disarray of positionalrelationships between respective color arrangements of RGB or the likecauses various image quality degradations in which a difference alsooccurs in colors between the output image and the original image.

CITATION LIST Patent Literature

PTL 1: Japanese Unexamined Patent Application Publication No.2008-278453

SUMMARY OF INVENTION Technical Problem

The present invention has been made, for example, in view of theabove-mentioned circumstances, and it is an object to provide an imagepickup apparatus and an image pickup apparatus control method, and aprogram with which it is possible to generate a high quality image inwhich an image quality degradation is suppressed when an image having adifferent number of pixels from the number of pixels provided to animage pickup element is generated and output.

Solution to Problem

A first aspect of the present invention resides in an image pickupapparatus including:

a pixel unit configured to output a pixel addition signal obtained byadding output pixel signals from pixels having different sensitivitiesto each other; and

a pixel information synthesis unit configured to execute a weightedaddition processing of multiplying plural pixel addition signals outputfrom the pixel unit by a previously set gain and calculate aconfiguration pixel value of a pixel number reduction image where atotal number of pixels is lower than a number of pixels of the pixelunit,

in which a control on gravity center positions for respective pixelsconstituting the pixel number reduction image is executed through thepixel addition signal generation processing by the pixel unit and theweighted addition processing by the pixel information synthesis unit.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the pixel unit and the pixel informationsynthesis unit execute a gravity center position control to evenlyarrange the gravity center positions for the respective pixelsconstituting the pixel number reduction image in the pixel unit.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the pixel unit executes a gravity center positioncontrol in one direction of the respective pixels constituting the pixelnumber reduction image in the pixel addition signal generationprocessing, and the pixel information synthesis unit executes, in theweighted addition processing, the gravity center position control in adirection orthogonal to the gravity center position control directionexecuted by the pixel unit.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the pixel unit performs a calculation for anaddition pixel value as a result of the gravity center position controlin accordance with a sensitivity ratio of pixels corresponding toaddition processing targets in the pixel addition signal generationprocessing.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the pixel information synthesis unit executes aweighted addition processing of multiplying plural pixel additionsignals corresponding to weighted addition processing targets by a gainin accordance with a gravity center position of a planned control in theweighted addition processing.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the pixel unit has a pixel array in which pixelsof a same color are arranged in every other row and pixels set to have adifferent sensitivity of a sensitivity ratio 1:3 are arranged every twoother rows, and the pixel unit has a configuration of outputting a pixeladdition signal obtained by adding output pixel signals from pixelshaving two different sensitivities where the sensitivity ratio is 1:3 toeach other.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the pixel unit has a pixel array in which pixelsof a same color are arranged in every other column, and the pixelinformation synthesis unit executes a weighted addition processing ofmultiplying two pixel addition signals output from the pixel unit inevery other column by a gain of 1:3 and calculates the configurationpixel value of the pixel number reduction image where the total numberof pixels is lower than the number of pixels of the pixel unit.

Furthermore, according to an embodiment of the image pickup apparatus ofthe present invention, the image pickup apparatus further includes acontrol unit configured to execute an exposure time control for units ofa region of the pixel unit.

Furthermore, a second aspect of the present invention resides in animage pickup apparatus control method executed in an image pickupapparatus, the control method including:

a pixel addition signal output step of outputting, in a pixel unit, apixel addition signal obtained by adding output pixel signals frompixels having different sensitivities to each other; and

a pixel information synthesis step of executing, in a pixel informationsynthesis unit, a weighted addition processing of multiplying pluralpixel addition signals output from the pixel unit by a previously setgain and calculating a configuration pixel value of a pixel numberreduction image where a total number of pixels is lower than a number ofpixels of the pixel unit,

in which a control on gravity center positions for respective pixelsconstituting the pixel number reduction image is executed through thepixel addition signal generation processing by the pixel unit and theweighted addition processing by the pixel information synthesis unit.

Furthermore, a third aspect of the present invention resides in aprogram for executing an image pickup apparatus control processing in animage pickup apparatus, the program including:

causing a pixel unit to execute a pixel addition signal output step ofoutputting a pixel addition signal obtained by adding output pixelsignals from pixels having different sensitivities to each other; and

causing a pixel information synthesis unit to execute a pixelinformation synthesis step of executing a weighted addition processingof multiplying plural pixel addition signals output from the pixel unitby a previously set gain and calculating a configuration pixel value ofa pixel number reduction image where a total number of pixels is lowerthan a number of pixels of the pixel unit,

in which a control on gravity center positions for respective pixelsconstituting the pixel number reduction image is executed through thepixel addition signal generation processing by the pixel unit and theweighted addition processing by the pixel information synthesis unit.

It should be noted that the program according to the present inventionis, for example, a program provided to an information processingapparatus or a computer system that can execute various program codes byway of, for example, a recording medium. A processing in accordance withthe program is realized while a program execution unit on theinformation processing apparatus or the computer system executes theabove-mentioned program.

Further objects, features, and advantages of the present invention willbecome apparent from the following detailed description based onembodiments of the present invention and the attached drawings. Itshould be noted that the system in the present specification is alogical aggregate structure of plural apparatuses and is not limited toa system in which apparatuses of the respective structures are locatedin a same casing.

Advantageous Effects of Invention

According to the embodiment of the present invention, the apparatus andmethod for realizing the processing with which the control on the pixelgravity center can be conducted when the pixel number reduction image isgenerated in the image pickup apparatus.

To be specific, a pixel unit configured to output a pixel additionsignal obtained by adding output pixel signals from pixels havingdifferent sensitivities to each other and a pixel information synthesisunit configured to execute a weighted addition processing of multiplyingplural pixel addition signals output from the pixel unit by a previouslyset gain and calculate a configuration pixel value of a pixel numberreduction image where a total number of pixels is lower than a number ofpixels of the pixel unit are provided. A control on gravity centerpositions for respective pixels constituting the pixel number reductionimage is executed through the pixel addition signal generationprocessing by the pixel unit and the weighted addition processing by thepixel information synthesis unit. For example, a gravity center positioncontrol to evenly arrange the gravity center positions for therespective pixels constituting the pixel number reduction image in thepixel unit is executed.

Through this processing, it is possible to generate a high quality imagein which a generation of jaggies or the like is suppressed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram for describing a pixel value additionprocessing in an image pickup element as a related art technology.

FIG. 2 is a timing chart in an image pickup processing by the imagepickup element.

FIG. 3 is an explanatory diagram for describing a problematic point in arelated art processing.

FIG. 4 is an explanatory diagram for describing a problematic point in arelated art processing.

FIG. 5 is an explanatory diagram for describing a generation of jaggiesas a problematic point in the related art processing.

FIG. 6 is an explanatory diagram for describing a configuration exampleof an image pickup apparatus.

FIG. 7 is an explanatory diagram for describing a configuration of theimage pickup element and a processing example.

FIG. 8 is an explanatory diagram for describing a generation example ofa pixel number reduction image.

FIG. 9 is an explanatory diagram for describing the pixel numberreduction image and a control on a pixel gravity center.

FIG. 10 is an explanatory diagram for describing the pixel numberreduction image and the control on the pixel gravity center.

FIG. 11 is an explanatory diagram for describing a detailedconfiguration of the image pickup element and a processing.

FIG. 12 is an explanatory diagram for describing a detailedconfiguration of the image pickup element and a processing.

FIG. 13 is an explanatory diagram for describing an exposure controlprocessing by the image pickup element.

FIG. 14 is an explanatory diagram for describing a gravity centerposition control in a vertical direction based on a pixel value additioncorresponding to a processing by the image pickup element.

FIG. 15 is an explanatory diagram for describing a block which is setwhen the pixel number reduction image is generated.

FIG. 16 is an explanatory diagram for describing the gravity centerposition control in the vertical direction based on the pixel valueaddition corresponding to the processing by the image pickup element.

FIG. 17 is an explanatory diagram for describing a gravity centerposition control in a horizontal direction corresponding to a processingby the image pickup element.

DESCRIPTION OF EMBODIMENTS

Hereinafter, details of an image pickup apparatus and an image pickupapparatus control method, and a program according to the presentinvention will be described with reference to the drawings. Thedescription will be conducted in accordance with the following items.

1. Regarding configuration example of image pickup apparatus

2. Regarding configuration example of image pickup element

3. Regarding specific example of generation processing for pixel numberreduction image

4. Regarding specific configuration and specific processing example ofimage pickup element

[1. Regarding Configuration Example of Image Pickup Apparatus]

First, an entire configuration example of an image pickup apparatus willbe described with reference to FIG. 6.

FIG. 6 is a block diagram illustrating a configuration example of theimage pickup apparatus. Incident light via an optical lens 101 enters animage pickup element 102 composed, for example, of a CMOS image sensoror the like, and image data based on a photoelectric conversion isoutput. The output image data is input to a signal processing unit 103.The signal processing unit 103 generates an output image 120 byexecuting, for example, a signal processing in a general camera such asa white balance (WB) adjustment or a gamma correction. The output image120 is stored in a storage unit that is not illustrated in the drawingor output to a display unit.

A control unit 105 outputs a control signal to the respective units inaccordance with, for example, a program stored in a memory that is notillustrated in the drawing and performs a control on variousprocessings.

[2. Regarding Configuration Example of Image Pickup Element]

Next, a configuration example of the image pickup element 102 will bedescribed with reference to FIG. 7.

FIG. 7 illustrates a configuration of the image pickup element 102according to an embodiment of the present invention.

The image pickup element 102 has a pixel unit 201 and an output pixelvalue calculation unit 202 as illustrated in FIG. 7.

The output pixel value calculation unit 202 has an AD conversion unit211, a pixel information synthesis unit 212, and an output unit 213.

The pixel unit 201 accumulates charges based on subject light in each ofa large number of pixels and outputs image data of a high number ofpixels corresponding to a high resolution image.

It should be noted that the pixel unit 201 is configured to have a highsensitivity pixel where a long time exposure is carried out and a lowsensitivity pixel where a short time exposure is carried out. A detailedconfiguration the pixel unit 201 will be described below.

The pixel unit has a pixel configuration composed of an array of RGbGrB,for example, similar to that described with reference to FIG. 1 aboveand is provided with an output circuit similar to FIG. 1( b), anaddition pixel value 250 obtained by two pixel values of a same color isoutput from the pixel unit 201.

It should be noted however that according to the present embodiment, theaddition pixel value 250 output from the pixel unit 201 is an additionvalue of the two pixel values of the same color, but an addition valueof two pixel values at these two different sensitivity is output as theaddition pixel value 250:

(a) the pixel value of the high sensitivity pixel where the long timeexposure is carried out, and

(b) the pixel value of the low sensitivity pixel where the short timeexposure is carried out.

A configuration of this processing will be described in a subsequentstage.

The addition pixel value 250 output from the pixel unit 201 is convertedinto digital data in the AD conversion unit 211 of the output pixelvalue calculation unit 202 and input to the pixel information synthesisunit 212.

In the pixel information synthesis unit 212, by utilizing digital dataon the addition pixel value input from the AD conversion unit 211, acalculation on pixel values of the respective pixels constituting theoutput image where the number of pixels in the pixel unit 201 isreduced.

For example, pixel values of respective pixels of a pixel numberreduction image where the number of pixels in the pixel unit 201 isreduced to ¼ are decided. A specific processing example will bedescribed below.

The output unit 213 outputs image data where the number of pixels isreduced which is input from the pixel information synthesis unit 212 tothe signal processing unit 103.

It should be noted that the control unit 105 supplies, for example, acontrol signal for an exposure time of the pixel unit 201, a processingcontrol signal for the output pixel value calculation unit 202, and thelike in accordance with the program stored in the memory that is notillustrated in the drawing.

[3. Regarding Specific Example of Generation Processing for Pixel NumberReduction Image]

Next, a specific example of a generation processing for a pixel numberreduction image which is executed by the image pickup apparatusaccording to the present embodiment will be described.

First, with reference to FIG. 8, a specific configuration example of thepixel unit 201 and an example of the pixel number reduction processingexecuted by the output pixel value calculation unit 202 will bedescribed.

FIG. 8 illustrates the following drawings.

(a) Pixel array example of the pixel unit 201

(b) Pixel number reduction image output by the image pickup element 102

In the drawing, to easily recognize pixel positions of the respectivepixels constituting the pixel unit, while the horizontal axis is set asx and the vertical axis is set as y, the respective pixel positions arerepresented by coordinates (x, y).

While the position of the Gb pixel on the upper left edge is set as (x,y)=(1, 1), and

the position of the Gr pixel on the lower right edge is set as (x,y)=(8, 8),

8×8 pixels are illustrated. It should be noted that the 8×8 pixelsillustrated in FIG. 8( a) is a drawing only illustrating a part of thepixel unit 201 (a part of the upper left edge part of the pixel unit201).

The pixel array of the pixel unit 201 illustrated in FIG. 8( a) has apixel array of Bayer pattern (RGbGrB) similarly as in the pixeldescribed above with reference to FIG. 1.

The pixel unit 201 has a configuration of including the high sensitivitypixel where the long time exposure is carried out and the lowsensitivity pixel where the short time exposure is carried out.

As illustrated in the drawing, a configuration is adopted from an upperrow in which,

the first row is a high sensitivity pixel row where the long timeexposure is carried out,

the second and third rows are low sensitivity pixel rows where the shorttime exposure is carried out,

the fourth and fifth rows are high sensitivity pixel rows where the longtime exposure is carried out, and

the sixth and seventh rows are low sensitivity pixel rows where theshort time exposure is carried out, and

hereinafter, the high sensitivity pixels and the low sensitivity pixelsare set in units of two rows.

It should be noted that these exposure time controls are executed, forexample, on the basis of the control signal of the control unit 105.

FIG. 8( b) illustrates a pixel number reduction image 280 output by theimage pickup element 201.

For example, on the basis of pixel values of Gb pixels on four cornersincluded in a 3×3 pixel block 271 illustrated on the upper left edge in(1) of FIG. 8, the image pickup element 102 decides and outputs a pixelvalue of one Gb pixel 281 on the upper left edge constituting the pixelnumber reduction image 280.

Similarly, for example, on the basis of pixel values of Gr pixels onfour corners included in a 3×3 pixel block 272 illustrated on the lowerright edge in (1) of FIG. 8, a pixel value of one Gr pixel 282 on thelower right edge constituting the pixel number reduction image 280 isdecided and output.

With regard to all the other respective pixels of RGbGrB, on the basisof pixel values of the same color on four corners of the 3×3 pixelblocks set at various positions, the image pickup element 102 decidesand outputs the respective pixel values of RGbGrB constituting the pixelnumber reduction image 280.

In this manner, the pixel values of the respective pixels of the pixelnumber reduction image 280 where the number of pixels in the pixel unit201 is reduced to ¼ are decided and output.

Next, with reference to FIG. 9, a setting example of a block (3×3 pixelblock) and a pixel gravity center of an output pixel (pixel set in thepixel number reduction image 280).

In the processing based on the addition processing of the plural pixelsof the image pickup element according to the related art technologydescribed above with reference to FIG. 1 to FIG. 4, the problem occursthat the gravity center position in the output image is shifted from theoriginal picked-up image.

In contrast to this, with the image pickup apparatus according to thepresent embodiment, a gravity center position control in the verticaldirection (y direction) is carried out by combining pixels withdifferent exposure times, that is, pixels having different sensitivitieswith each other, and further a weighted addition processing where a gain(weight) is adjusted is carried out to conduct a gravity center positioncontrol in the horizontal direction (x direction). Through theseprocessings, relative positions of the respective colors set in thepixel number reduction image corresponding to the output image can beoutput as the same setting on the relative positions of the gravitycenter position of the respective colors in the original picked-upimage.

To be specific, as illustrated in FIG. 9( b), the pixel number reductionimage with a setting of arranging the gravity center positions where therespective colors of RGbGrB are evenly arranged is generated.

On the basis of Gb pixels on four corners of a 3×3 pixel block 321illustrated in FIG. 9( a), a Gb pixel value having a gravity centerposition at a Gb pixel 331 illustrated in FIG. 9( b) is calculated.

On the basis of B pixels on four corners of a 3×3 pixel block 322illustrated in FIG. 9( a), a B pixel value having a gravity centerposition at a B pixel 332 illustrated in FIG. 9( b) is calculated.

On the basis of Gb pixels on four corners of a 3×3 pixel block 323illustrated in FIG. 9( a), a Gb pixel value having a gravity centerposition at a Gb pixel 333 illustrated in FIG. 9( b) is calculated.

On the basis of B pixels on four corners of a 3×3 pixel block 324illustrated in FIG. 9( a), a B pixel value having a gravity centerposition at a B pixel 334 illustrated in FIG. 9( b) is calculated.

With regard to all the other RGbGrB pixels, a 3×3 pixel block is set inthe pixel array of the image pickup element illustrated in FIG. 9( a),and on the basis of pixel values of the pixels having the same color onfour corners in the block, a pixel value with a setting in which agravity center is located at a position illustrated in FIG. 9( b) iscalculated.

It should be noted that according to the related art example describedabove with reference to FIG. 1 to FIG. 4 and also the present embodimenttoo, the setting mode on the 3×3 pixel block illustrated in FIG. 9( a)is the same.

According to the related art example described with reference to FIG. 1to FIG. 4, since all the pixels have the setting of the same sensitivity(same exposure time), a pixel gravity center of one pixel calculatedfrom the pixel values of the pixels on the four corner in the 3×3 pixelblock is set at the center of the 3×3 pixel block, which corresponds topositions of a pixel Gb 341 to a pixel B 344 illustrated in FIG. 9( b).This is the same as described above with reference to FIG. 3 and FIG. 4.

In contrast to this, according to the present embodiment, the gravitycenter position control for setting gravity center positions of therespective pixels after the reduction in the number of pixels at evenlyarranged positions in the horizontal direction and also the verticaldirection illustrated in FIG. 9( b) is carried out.

The control on the pixel gravity center is carried out through thefollowing synthesis processings on plural pixels.

These synthesis processings are executed to control the gravity centerposition:

(a) a synthesis processing on pixels having different sensitivitydifferences for the gravity center position control in the verticaldirection (y direction), and

(b) a synthesis processing of setting a gain (weight) for the gravitycenter position control in the horizontal direction (x direction).

A specific control example on the position of the pixel gravity centeraccording to the present embodiment will be described with reference toFIG. 10.

FIG. 10(A) illustrates

four patterns of 3×3 pixel blocks set in a pixel array of the imagepickup element. These respective blocks are illustrated:

(a1) a pixel block 351 having Gb pixels on four corners,

(a2) a pixel block 352 having B pixels on four corners,

(a3) a pixel block 353 having R pixels on four corners, and

(a4) a pixel block 354 having Gr pixels on four corners.

Furthermore, these are illustrated in FIG. 10(B):

(b1) a position of a pixel gravity center 371 of the Gb pixels of thepixel number reduction image generated on the basis of the four Gbpixels in the pixel block 351,

(b2) a position of a pixel gravity center 372 of the B pixels of thepixel number reduction image generated on the basis of the four B pixelsin the pixel block 352,

(b3) a position of a pixel gravity center 373 of the R pixels of thepixel number reduction image generated on the basis of the four R pixelsin the pixel block 353, and

(b4) a position of a pixel gravity center 374 of the Gr pixels of thepixel number reduction image generated on the basis of the four Grpixels in the pixel block 354.

As illustrated in FIG. 10(B), set position modes for the pixel gravitycenter are different from each other in units of each color of therespective colors: Gb, B, R, and Gr as follows.

(b1) The pixel gravity center 371 of the Gb pixels of the pixel numberreduction image generated on the basis of the four Gb pixels in thepixel block 351 is set as an upper left edge part of the center pixel ofthe pixel block 351.

(b2) The pixel gravity center 372 of the B pixels of the pixel numberreduction image generated on the basis of the four B pixels in the pixelblock 352 is set as a right edge part of the center pixel of the pixelblock 352.

(b3) The pixel gravity center 373 of the R pixels of the pixel numberreduction image generated on the basis of the four R pixels in the pixelblock 353 is set as a lower left edge part of the center pixel of thepixel block 353.

(b4) The pixel gravity center 374 of the Gr pixels of the pixel numberreduction image generated on the basis of the four Gr pixels in thepixel block 354 is set as a lower right edge part of the center pixel ofthe pixel block 354.

In any of these colors, the point of calculating the pixel value of onepixel set in the pixel number reduction image by utilizing the pixelvalues of the pixels having the same color on the four corners of the3×3 pixel block is the same, but a different setting is made for each ofthese colors for the position of the pixel gravity center which is seton the basis of these four pixels.

In the image pickup apparatus according to the present embodiment, asillustrated in FIG. 10(B), the gravity center position control iscarried out in a different mode for each of the respective colors. Thatis, the pixel value calculation utilizing the four pixel values of the3×3 pixel block is executed as a different processing in units of eachcolor.

It should be noted that the control for the gravity center position isbasically carried out in a combination of the following two processings.The control is executed by these processings:

(a) a synthesis processing on pixels with different sensitivitydifferences for a shift of the gravity center position in the verticaldirection (y direction), and

(b) a synthesis processing in which a gain (weight) is set for a shiftof the gravity center position in the horizontal direction (xdirection).

Through these pixel syntheses, a configuration is established in whichthe arrangement of the respective colors for the constitutional pixelsof the pixel number reduction image is an even arrangement illustratedin FIG. 9( b), that is, the gravity center positions of all the colorare arranged at even intervals both in the horizontal direction and thevertical direction.

In a case where these evenly arranged pixels are integrated and outputas the pixel number reduction image composed of the ¼ number of pixelsfor the picked-up image of the original image pickup element, andcompression rates for distances between the respective pixelsconstituting the pixel number reduction image are all the samecompression rate.

As a result, it is possible to generate and output the image that doesnot cause the decrease in the image quality such as the expansion ofjaggies or color blurring described above with reference to FIG. 5.

[4. Regarding Specific Configuration and Specific Processing Example ofImage Pickup Element]

Next, a specific configuration and processing of the image pickupelement that realizes the above-mentioned control processing on thegravity center position.

FIG. 11 illustrates the respective constructional units of the imagepickup element 102 illustrated in FIG. 7 as well as the specificprocessing example.

FIG. 11 illustrates the pixel unit 201 and the output pixel valuecalculation unit 202 similar to those in FIG. 7.

The pixel value calculation unit 202 has the AD conversion unit 211, thepixel information synthesis unit 212, and the output unit 213.

The pixel unit 201 has Bayer pattern of the RGbGrB pattern. This pixelunit 201 outputs a pixel value obtained by adding two pixels of a samecolor in the vertical direction (y direction) in every other row to eachother.

A circuit for adding two pixels having a same color in the verticaldirection (y direction) with to each other will be described withreference to FIG. 12. In the pixel unit, in accordance with a circuitconfiguration illustrated in FIG. 12( b), a pixel value obtained byadding the two pixels having the same color in the vertical direction (ydirection) in every other row to each other is output.

The circuit illustrated in FIG. 12( b) is a circuit similar to thatdescribed above with reference to FIG. 1( b).

The circuit illustrated in FIG. 12( b) is a circuit corresponding to apixel area 301 for four pixels of Gb1, R1, Gb2, and R2 illustrated inFIG. 12( a). The pixels of Gb and R are connected via transfertransistors (T1 to T4) to a gate part of a transistor for amplification(AMP) as illustrated in FIG. 12( b).

By using the circuit configuration illustrated in FIG. 12( b), anaddition computation is carried out on pixels having a same colorincluded in the pixel area 301 (according to the present example, Gb1and Gb2).

In the read out processing for the pixel value, the selection (SEL) ofthe common pixel and the reset (RST) of the floating diffusion (FD) arecarried out, the transistors T1 and T3 in the pixel Gb1 and the pixelGb2 are simultaneously read out, and electrons generated in Gb1 and Gb2are accumulated in the floating diffusion (FD) to be added to eachother. Thus, an addition signal based on pixel values of the two pixelsis obtained.

Through this addition processing, the addition signal based on the pixelvalues of the two pixels in the vertical direction is output from thepixel unit 201.

It should be noted however that according to the present embodiment, thepixels corresponding to the addition targets include

(a) the high sensitivity pixel where the long time exposure is carriedout, and

(b) the low sensitivity pixel where the short time exposure is carriedout,

and the two pixels have these different sensitivities. According to thepresent embodiment, the pixel value addition processing at these twodifferent sensitivities is executed.

In the pixel unit 201 of the image pickup element, the high sensitivitypixels and the low sensitivity pixels are set in units of two rows asillustrated in FIG. 12( a), and according to the present embodiment,among the pixels on the four corner of the 3×3 pixel block set as thebasis unit for calculating the synthesis pixel value, the two pixelshaving the same color arranged in the vertical direction (y direction)certainly have a combination of these pixels having the differentsensitivities:

(a) the high sensitivity pixel where the long time exposure is carriedout, and

(b) the low sensitivity pixel where the short time exposure is carriedout.

As a result, the pixel addition is executed as this pixel additionprocessing of the different sensitivities.

A data example output from an output line illustrated below the pixelunit 201 of FIG. 11 is illustrated in step (S1) of FIG. 11.

Herein, as a representative example, a synthesis processing example onGb pixels included in a 3×3 pixel block 401 illustrated in FIG. 11 isillustrated as step (S1) and step (S2).

Step (S1) illustrated in FIG. 11 is a synthesis processing on pixelshaving different sensitivity differences and corresponds to the controlprocessing on the gravity center position in the vertical direction (ydirection).

Step (S2) is a synthesis processing in which the gain (weight) is setand corresponds to the control processing on the gravity center positionin the horizontal direction (x direction).

A sensitivity difference (exposure time ratio) between the highsensitivity and the low sensitivity is set as 3:1.

FIG. 13 illustrates an exposure control processing example by the imagepickup element 102.

The first row is the high sensitivity (long time exposure) pixel row.

The second and third rows are the low sensitivity (short time exposure)pixel rows.

The fourth and fifth rows are the high sensitivity (long time exposure)pixel rows.

The sixth and seventh rows are the low sensitivity (short time exposure)pixel rows.

Hereinafter, the high sensitivity and low sensitivity pixel rows arerepeatedly set in units of two rows.

An exposure time control processing by the pixel unit 201 will bedescribed with reference to FIG. 13.

With respect to the high sensitivity pixel, along a shutter startposition S1 illustrated in FIG. 13, the exposure is sequentially startedfrom the upper row towards the lower row.

On the other hand, with respect to the high sensitivity pixel, along ashutter start position S2 illustrated in FIG. 13, the exposure issequentially started from the upper row towards the lower row.

In the read out processing, with respect to all the rows, the read outis sequentially executed along a read out position E1 on all the rowsfrom the upper row to the lower row.

The control on this processing is executed, for example, on the basis ofthe control signal from the control unit 105.

As a result of this exposure control, the exposure processing in whichthe sensitivity difference (exposure time ratio) between the highsensitivity and the low sensitivity is set as 3:1 is executed.

In the 3×3 pixel block set in the pixel unit 201 composed of the twopixel rows with the different exposure times, the pixels having the samecolor on the upper edge row and the lower edge row are certainly a pairof the high sensitivity pixels and the low sensitivity pixels.

Therefore, the pixels corresponding to the addition targets of the twopixel addition circuit illustrated in FIG. 12( b) are a combination ofthe high sensitivity pixel and the low sensitivity pixel.

As a result, as illustrated in step (S1) of FIG. 11, the output from thepixel unit 201 illustrated in FIG. 11 is output as the pixel value atthe position where the pixel gravity center approaches the highsensitivity pixel side.

A detail of the gravity center position control processing in thisvertical direction (y direction) will be described with reference toFIG. 14.

FIG. 14 is an explanatory diagram for describing a gravity centerposition in the two pixel addition for the same color pixels in thevertical direction (y direction) and illustrates these two examples:

(a) a two pixel addition for the pixels having the same sensitivity, and

(b) a two pixel addition for the pixels having the differentsensitivities (sensitivity ratio 3:1).

FIG. 14( a) illustrates the gravity center position in the two pixeladdition for the same color pixels in the vertical direction (ydirection) in the case of the setting for the same sensitivity (sameexposure time) in all the pixels described above with reference to FIG.1 to FIG. 4.

In this case, the gravity center is set at a center position of thepixel positions for the two pixels corresponding to the addition targets(Gb1 and Gb2).

FIG. 14( b) corresponds to the image pickup apparatus according to thepresent embodiment and the case of the two pixel addition processing forthe pixels having the different sensitivities (sensitivity ratio 3:1),and in this case, the gravity center position is set at a position closeto the high sensitivity pixel (Gb1) side.

A ratio of a distance from the high sensitivity pixel (Gb1) to thegravity center to a distance from the low sensitivity pixel (Gb2) to thegravity center takes an inverse number of the sensitivity ratio and is1:3.

That is, the gravity center is set at a position close to the pixelposition of the high sensitivity pixel (Gb1).

In this manner, the pixel gravity center in the vertical direction iscontrolled.

This processing is represented by the processing in step (S1) of FIG.11.

The output of the pixel unit 201 illustrated in FIG. 11 is output as avalue on which the control in the vertical direction for the pixelgravity center is already carried out.

It should be noted that as described with reference to FIG. 10(B),

the gravity center position control mode is varied for each of therespective colors.

As being understood from FIG. 10(B),

the gravity center position in the vertical direction needs to be set onthe upper side of the center pixel for the 3×3 pixel block with regardto Gb and B.

With regard to R and Gr, the gravity center position in the verticaldirection needs to be set on the lower side of the center pixel for the3×3 pixel block.

To realize this processing,

the 3×3 pixel block for calculating Gb and B is a block where the highsensitivity pixel is set on the upper side like the pixel block 401illustrated in FIG. 11. With this block setting, the gravity center isset while being shifted towards the upper side.

On the other hand, the 3×3 pixel block for calculating R and Gr is ablock where the high sensitivity pixel is set on the lower side like apixel block 402 illustrated in FIG. 15. With this block setting, thegravity center is set while being shifted towards the lower side.

FIG. 16 illustrates a correspondence relationship between the blocks inaccordance with the respective colors and the sensitivity settings, andthe gravity center positions in the vertical direction.

(1) Processing example with respect to Gb and B

(2) Processing example with respect to R and Gr

These processing examples are illustrated.

As illustrated in FIG. 16 (1), in a case where Gb and B are calculated,the 3×3 pixel block for calculating Gb and B is a block where the highsensitivity pixel is set on the upper side. With this block setting, thegravity center is set while being shifted towards the upper side.

On the other hand, as illustrated in FIG. 16 (2), in a case where R andGr are calculated, the 3×3 pixel block for calculating R and Gr is ablock where the high sensitivity pixel is set on the lower side. Withthis block setting, the gravity center is set while being shiftedtowards the lower side.

The AD conversion unit 211 of the output pixel value calculation unit202 illustrated in FIG. 11 executes the AD conversion on this additionvalue to generate a digital value.

This is the AD conversion unit output value (pixel addition value) 291illustrated in FIG. 11.

This AD conversion unit output value (pixel addition value) 291 is inputto the pixel information synthesis unit 212.

The pixel information synthesis unit 212 performs

the control for the gravity center position in the horizontal direction(x direction), and this processing is realized through the synthesisprocessing in which the gain (weight) is set.

With regard to this processing, the processing on the Gb pixel of the3×3 pixel block 401 illustrated in FIG. 11 will be described as anexample.

The pixel information synthesis unit 212 utilizes two addition values,that is, these two Gb addition values:

[Gb12Data] corresponding to an addition value output of Gb1 and Gb2 inthe 3×3 pixel block 401, and

[Gb34Data] corresponding to an addition value output of Gb3 and Gb4 inthe 3×3 pixel block 401,

and the final Gb pixel value set in the pixel number reduction image iscalculated.

This pixel value calculation processing is carried out through theprocessing taking into account the pixel gravity center position, thatis, the pixel value computation processing accompanied by the weightedaddition in which the gain (weight) is set.

This processing is represented in step (S2) illustrated in FIG. 11.

As described above with reference to FIG. 10( b 1),

the gravity center position for the Gb pixels needs to be set at anupper left edge position of the center pixel of the 3×3 pixel block 351illustrated in FIG. 10( b 1).

For the processing for this purpose, the calculation on the pixel value(Gb(out)) of the Gb pixel of the pixel number reduction image isexecuted in accordance with the following expression (Expression 1).

Gb(out)=((Gb12×3)+Gb34)/4  (Expression 1)

It should be noted that in the above-mentioned expression,

Gb12 denotes a pixel addition value of Gb1 and Gb2, and

Gb34 denotes a pixel addition value of Gb3 and Gb4.

Through the pixel value calculation in accordance with theabove-mentioned expression (Expression 1), the gravity center positionis set at the upper left edge of the center pixel of the 3×3 pixel blockas illustrated in step (S2) of FIG. 11.

This result corresponds to the position of the pixel gravity center 371for the position of the Gb pixels illustrated in FIG. 10( b 1).

With regard to the Gb pixel, upon the computation of the two additionpixel values, a computation of multiplying the addition value on theleft side (according to the present example, Gb12) by a larger gain(weight) than that for the addition value on the right side (Gb34) isexecuted so that the gravity center is shifted towards the left sidefrom the center of the two addition pixel positions.

This gain (weight) setting mode varies for each of the respectivecolors.

As is apparent from the drawing illustrated in FIG. 10(B),

with regard to the Gb pixel and the R pixel, a computation ofmultiplying the addition value on the left side by a larger gain(weight) than that for the addition value on the right side is executedso that the gravity center is shifted towards the left side from thecenter of the two addition pixel positions.

On the other hand,

with regard to the B pixel and the Gr pixel, so that a computation ofmultiplying the addition value on the right side by a larger gain(weight) than that for the addition value on the left side is executedthe gravity center is shifted towards the right side from the center ofthe two addition pixel positions.

Through the above-mentioned processing, the gravity center positioncontrol in the horizontal direction is carried out.

It should be noted that the gain ratio is 3:1 in the present example.

FIG. 17 illustrates a correspondence relationship between the blocks inaccordance with the respective colors and the sensitivity settings, andthe gravity center positions in the horizontal direction.

(1) Processing example with respect to Gb and R

(2) Processing example with respect to B and Gr

These processing examples are illustrated.

As illustrated in FIG. 17 (1), in a case where Gb and R are calculated,among the two addition values obtained from the 3×3 pixel block forcalculating Gb and R, the gain is set as 3 for the addition value on theleft side, and the gain is set as 1 for the addition value on the rightside, so that through the weighted addition processing based on thisgain setting, the gravity center can be set while being shifted towardsthe left side.

On the other hand, as illustrated in FIG. 17 (2), in a case where B andGr are calculated, among the two addition values obtained from the 3×3pixel block for calculating B and Gr, the gain is set as 1 for theaddition value on the left side, and the gain is set as 3 for theaddition value on the right side, so that through the weighted additionprocessing based on this gain setting, the gravity center can be setwhile being shifted towards the right side.

In the pixel information synthesis unit 212 illustrated in FIG. 11, byexecuting the pixel value calculation processing accompanied by theabove-mentioned gravity center position control in the horizontaldirection, one output pixel value is decided on the basis of the twoaddition values.

This result is output to the signal processing unit via the output unit213.

As described above with reference to FIG. 9( b), the pixel gravitycenters for the respective constitutional pixels of the pixel numberreduction image generated by the gravity center position control have asetting in which the respective pixel positions are evenly arranged, andthe pixel number reduction image having the relative pixel positionsimilar to the pixel arrangement of the picked-up image of the originalimage pickup element is obtained.

As a result, the high quality image in which the generation of jaggiesor the like described above with reference to FIG. 5 is suppressed isoutput.

It should be noted that according to the above-mentioned embodiment, theconfiguration has been described in which the 3×3 pixel block is set asthe unit and the sensitivity difference in units of two rows is set as3:1, but other various settings can be made for the block size, thesensitivity ratio, and the like. If the gain setting and the like areadjusted in accordance with the respective settings to carry out theprocessing, it is possible to conduct the control for setting thegravity center position in various positions.

In the above, the present invention has been described in detail withreference to the particular embodiments. However, it is apparent thatthose skilled in the art may make various modifications and alterationsof the embodiments within a scope without departing from the gist of thepresent invention. That is, the present invention has been disclosed byway of exemplification and should be not construed in a limited manner.To determine the gist of the present invention, the section of claimsshould be considered.

Also, the series of processings described in the specification can beexecuted by hardware or software, or a composite configuration of those.In a case where the series of processings is executed by the software, aprogram recording a processing sequence can be installed into a memoryin a computer which is incorporated in dedicated-use hardware for theexecution, or the program can be installed into a general-use computerthat can execute various processings for the execution. For example, theprogram can be previously recorded in a recording medium. In addition tothe installment from the recording medium into the computer, the programcan be received via a network such as LAN (Local Area Network) or theinternet and installed into the recording medium such as a built-in harddisc.

It should be noted that the various processings described in thespecification may be executed not only in a time-series manner inaccordance with the description but also executed in parallel orindividually in accordance with a processing performance of theapparatus that executes the processing or as appropriate. Also, thesystem in the present specification refers to a logical aggregatestructure of plural apparatuses and is not limited to a system in whichthe apparatuses of the respective configurations are located in a samecasing.

INDUSTRIAL APPLICABILITY

As described above, according to the embodiment of the presentinvention, the apparatus and method for realizing the processing withwhich it is possible to conduct the control on the pixel gravity centerwhen the pixel number reduction image is generated in the image pickupapparatus are realized.

To be specific, a pixel unit configured to output a pixel additionsignal obtained by adding output pixel signals from pixels havingdifferent sensitivities to each other and a pixel information synthesisunit configured to execute a weighted addition processing of multiplyingplural pixel addition signals output from the pixel unit by a previouslyset gain and calculate a configuration pixel value of a pixel numberreduction image where a total number of pixels is lower than a number ofpixels of the pixel unit are provided. A control on gravity centerpositions for respective pixels constituting the pixel number reductionimage is executed through the pixel addition signal generationprocessing by the pixel unit and the weighted addition processing by thepixel information synthesis unit. For example, a gravity center positioncontrol is executed so as to evenly arrange the gravity center positionsfor the respective pixels constituting the pixel number reduction imagein the pixel unit.

Through this processing, it is possible to generate a high quality imagein which a generation of jaggies or the like is suppressed.

REFERENCE SIGNS LIST

11 PIXEL REGION

21 to 24 BLOCK

31 to 34 PIXEL

101 OPTICAL LENS

102 IMAGE PICKUP ELEMENT

103 SIGNAL PROCESSING UNIT

105 CONTROL UNIT

120 OUTPUT IMAGE

201 PIXEL UNIT

202 OUTPUT PIXEL VALUE CALCULATION UNIT

211 AD CONVERSION UNIT

212 PIXEL INFORMATION SYNTHESIS UNIT

213 OUTPUT UNIT

250 ADDITION PIXEL VALUE

291 AD CONVERSION UNIT OUTPUT VALUE (PIXEL ADDITION VALUE)

321 to 324 BLOCK

331 to 334 PIXEL

351 to 354 BLOCK

371 to 374 PIXEL GRAVITY CENTER

401 BLOCK

421 BLOCK

1. An image pickup apparatus comprising: a pixel unit configured tooutput a pixel addition signal obtained by adding output pixel signalsfrom pixels having different sensitivities to each other; and a pixelinformation synthesis unit configured to execute a weighted additionprocessing of multiplying plural pixel addition signals output from thepixel unit by a previously set gain and calculate a configuration pixelvalue of a pixel number reduction image where a total number of pixelsis lower than a number of pixels of the pixel unit, wherein a control ongravity center positions for respective pixels constituting the pixelnumber reduction image is executed through the pixel addition signalgeneration processing by the pixel unit and the weighted additionprocessing by the pixel information synthesis unit.
 2. The image pickupapparatus according to claim 1, wherein the pixel unit and the pixelinformation synthesis unit execute a gravity center position control toevenly arrange the gravity center positions for the respective pixelsconstituting the pixel number reduction image in the pixel unit.
 3. Theimage pickup apparatus according to claim 1, wherein the pixel unitexecutes a gravity center position control in one direction of therespective pixels constituting the pixel number reduction image in thepixel addition signal generation processing, and wherein the pixelinformation synthesis unit executes, in the weighted additionprocessing, the gravity center position control in a directionorthogonal to the gravity center position control direction executed bythe pixel unit.
 4. The image pickup apparatus according to claim 1,wherein the pixel unit performs a calculation for an addition pixelvalue as a result of the gravity center position control in accordancewith a sensitivity ratio of pixels corresponding to addition processingtargets in the pixel addition signal generation processing.
 5. The imagepickup apparatus according to claim 1, wherein the pixel informationsynthesis unit executes a weighted addition processing of multiplyingplural pixel addition signals corresponding to weighted additionprocessing targets by a gain in accordance with a gravity centerposition of a planned control in the weighted addition processing. 6.The image pickup apparatus according to claim 1, wherein the pixel unithas a pixel array in which pixels of a same color are arranged in everyother row and pixels set to have a different sensitivity of asensitivity ratio 1:3 are arranged every two other rows, and wherein thepixel unit has a configuration of outputting a pixel addition signalobtained by adding output pixel signals from pixels having two differentsensitivities where the sensitivity ratio is 1:3 to each other.
 7. Theimage pickup apparatus according to claim 1, wherein the pixel unit hasa pixel array in which pixels of a same color are arranged in everyother column, and wherein the pixel information synthesis unit executesa weighted addition processing of multiplying two pixel addition signalsoutput from the pixel unit in every other column by a gain of 1:3 andcalculates the configuration pixel value of the pixel number reductionimage where the total number of pixels is lower than the number ofpixels of the pixel unit.
 8. The image pickup apparatus according toclaim 1, further comprising: a control unit configured to execute anexposure time control for units of a region of the pixel unit.
 9. Animage pickup apparatus control method executed in an image pickupapparatus, the control method comprising: a pixel addition signal outputstep of outputting, in a pixel unit, a pixel addition signal obtained byadding output pixel signals from pixels having different sensitivitiesto each other; and a pixel information synthesis step of executing, in apixel information synthesis unit, a weighted addition processing ofmultiplying plural pixel addition signals output from the pixel unit bya previously set gain and calculating a configuration pixel value of apixel number reduction image where a total number of pixels is lowerthan a number of pixels of the pixel unit, wherein a control on gravitycenter positions for respective pixels constituting the pixel numberreduction image is executed through the pixel addition signal generationprocessing by the pixel unit and the weighted addition processing by thepixel information synthesis unit.
 10. A program for executing an imagepickup apparatus control processing in an image pickup apparatus, theprogram comprising: causing a pixel unit to execute a pixel additionsignal output step of outputting a pixel addition signal obtained byadding output pixel signals from pixels having different sensitivitiesto each other; and causing a pixel information synthesis unit to executea pixel information synthesis step of executing a weighted additionprocessing of multiplying plural pixel addition signals output from thepixel unit by a previously set gain and calculating a configurationpixel value of a pixel number reduction image where a total number ofpixels is lower than a number of pixels of the pixel unit, wherein acontrol on gravity center positions for respective pixels constitutingthe pixel number reduction image is executed through the pixel additionsignal generation processing by the pixel unit and the weighted additionprocessing by the pixel information synthesis unit.